Convolutional Neural Network

In this second exercise-notebook we will play with Convolutional Neural Network (CNN).

As you should have seen, a CNN is a feed-forward neural network tipically composed of Convolutional, MaxPooling and Dense layers.

If the task implemented by the CNN is a classification task, the last Dense layer should use the Softmax activation, and the loss should be the categorical crossentropy.

Reference: https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py

Training the network

We will train our network on the CIFAR10 dataset, which contains 50,000 32x32 color training images, labeled over 10 categories, and 10,000 test images.

As this dataset is also included in Keras datasets, we just ask the keras.datasets module for the dataset.

Training and test images are normalized to lie in the $\left[0,1\right]$ interval.



In [6]:

    
from keras.datasets import cifar10
from keras.utils import np_utils

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
X_train = X_train.astype("float32")
X_test = X_test.astype("float32")
X_train /= 255
X_test /= 255

To reduce the risk of overfitting, we also apply some image transformation, like rotations, shifts and flips. All these can be easily implemented using the Keras Image Data Generator.

Warning: The following cells may be computational Intensive....



In [7]:

    
from keras.preprocessing.image import ImageDataGenerator

generated_images = ImageDataGenerator(
    featurewise_center=True,  # set input mean to 0 over the dataset
    samplewise_center=False,  # set each sample mean to 0
    featurewise_std_normalization=True,  # divide inputs by std of the dataset
    samplewise_std_normalization=False,  # divide each input by its std
    zca_whitening=False,  # apply ZCA whitening
    rotation_range=0,  # randomly rotate images in the range (degrees, 0 to 180)
    width_shift_range=0.2,  # randomly shift images horizontally (fraction of total width)
    height_shift_range=0.2,  # randomly shift images vertically (fraction of total height)
    horizontal_flip=True,  # randomly flip images
    vertical_flip=False)  # randomly flip images

generated_images.fit(X_train)

Now we can start training.

At each iteration, a batch of 500 images is requested to the ImageDataGenerator object, and then fed to the network.



In [10]:

    
X_train.shape









    Out[10]:





(50000, 3, 32, 32)



In [11]:

    
gen = generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True)
X_batch, Y_batch = next(gen)



In [12]:

    
X_batch.shape









    Out[12]:





(500, 3, 32, 32)



In [ ]:

    
from keras.utils import generic_utils

n_epochs = 2
for e in range(n_epochs):
    print('Epoch', e)
    print('Training...')
    progbar = generic_utils.Progbar(X_train.shape[0])
    
    for X_batch, Y_batch in generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True):
        loss = model.train_on_batch(X_batch, Y_batch)
        progbar.add(X_batch.shape[0], values=[('train loss', loss[0])])